Personal sensory drones

ABSTRACT

Various systems and methods for personal sensory drones are described herein. A personal sensory drone system includes a drone remote control system comprising: a task module to transmit a task to a drone swarm for the drone swarm to execute, the drone swarm including at least two drones; a transceiver to receive information from the drone swarm related to the task; and a user interface module to present a user interface based on the information received from the drone swarm.

TECHNICAL FIELD

Embodiments described herein generally relate to sensory augmentationdevices and in particular, to personal sensory drones.

BACKGROUND

Aerial drones or unmanned aerial vehicles (UAV) have become more popularin recent years due to the reduction in component cost and increase ininterest. Manufacturing and miniaturization continue to advance dronetechnology and produce smaller, lighter drones with more capabilities.Several military and civil applications have been developed, includingreconnaissance, border patrolling, crop surveying, filmmaking, searchand rescue operations, and parcel delivery.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. Like numerals havingdifferent letter suffixes may represent different instances of similarcomponents. Some embodiments are illustrated by way of example, and notlimitation, in the figures of the accompanying drawings in which:

FIG. 1 is a diagram illustrating components of a drone and a remotecontrol system, according to an embodiment;

FIGS. 2-4 are flowcharts illustrating control and data flow duringoperation, according to an embodiment;

FIG. 5 is a diagram illustrating operation of a drone, according to anembodiment;

FIG. 6 is a block diagram illustrating a personal sensory drone system,according to an embodiment;

FIG. 7 is a flowchart illustrating a method of implementing a personalsensory drone system, according to an embodiment; and

FIG. 8 is a block diagram illustrating an example machine upon which anyone or more of the techniques (e.g., methodologies) discussed herein mayperform, according to an example embodiment.

DETAILED DESCRIPTION

Systems and methods described herein provide a system for personalsensory drones. Drones and other robots constitute a new technology thatmay disrupt life in many aspects, from surveillance and security toshipments, disaster aid, work tasks, military applications, etc. Manypeople could benefit from drones that enable remote sensingcapabilities. For example, people with vision impairment, mobilityissues, or hearing problems may benefit from remote sensing providing bydrones.

The present disclosure discusses an improvement to the operation ofautonomous unmanned vehicles. Drones that are personal to a user may beused in a swarm or individually to provide content to the user. In aswarm, drones may be synchronized to provide an integrated sensoryenvironment to the user. For example, several drones may stitch togethervideo so that a user may view the video feeds as if they were a singlefeed. The user may have a virtual reality headset or other panoramicvision system to experience a wide angle view (e.g., 360 degree view) ofan object or a scene. Alternatively, several drones may be used toprovide different fields of view (FOV), perspectives, or video feeds(e.g., infrared and visible light views) to a user who may togglebetween different views or experience several views side by side in anaggregated display. Similar mechanisms may be provided for other sensessuch as hearing, touching, or smelling. The drones may also allow a formof remote sensing so that a user may have smells and hapticrepresentations brought back by the drone. This may also assist userswith mobility problems that prevent them from walking up to the objectof interest. In effect, personal drones may augment a user's senses.

FIG. 1 is a diagram illustrating components of a drone 100 and a remotecontrol system 102, according to an embodiment. The drone 100 may alsobe referred to as a semi-autonomous vehicle or robot, which may beland-based, maritime vehicles, unmanned aerial vehicle (UAVs), or thelike. For discussion purposes, the drone 100 discussed herein is anunmanned aerial vehicle (UAV). The remote control system 102 may presentone or more user interfaces (UI) to one or more users and be used tocontrol the drone 100. For example, a UI may display a map of an areaand allow a user to select waypoints that the drone 100 may follow.Communication between the remote control system 102 and the drone 100 isbi-directional. As such, images and other information captured by thedrone 100 may be transmitted back to the remote control system 102 fordisplay to the user, who may then respond with additional commands orother parameter changes.

The drone 100 includes a flight module 104, flight hardware 106, aflight map 108, a sensor array 110, and a communication subsystem 112.The drone 100 may operate semi-autonomously based on commands receivedfrom the remote control system 102. For example, when the drone 100receives a navigation command that includes a destination—such as GPScoordinates and desired altitude—the drone 100 may move to thedestination without further user input. The drone 100 may also includeoutput devices 124, such as a display or speakers.

The flight hardware 106 includes the components of the drone 100 thatpropel or otherwise move the drone 100. For example, the flight hardware106 for a quadrotor helicopter UAV (also known as a quadcopter), mayinclude four propellers. The flight hardware 106 may vary depending onthe type of drone 100 (e.g., wheels for a ground-based unit). The flighthardware 106 may also include a GPS receiver. The GPS receiver may becapable of differential GPS (DGPS) or real time kinematic (RTK) GPS. Theflight hardware 106 may also include at least one processing unit (e.g.,a central processing unit, a graphical processor, or anapplication-specific integrated circuit). The processing unit mayexecute software stored on the drone 100 to perform the functionsdescribed herein of the drone 100.

The flight map 108 includes data representing a geographic area, whichmay include roads, arena seating, audience stands, stages, fields,playing surfaces, or the like, and GPS coordinates of various featuresof these geographic areas. The flight map 108 includes altitude data ofthe geographic area. The data may also include location data of man-madeobjects such as bridges, cell towers, etc. Furthermore, the flight map108 may include a database of point-of-interest (POI) locationsincluding, but not limited to restaurants, businesses, gas stations,stadiums, golf courses, etc. It is understood that the flight map 108may include additional geographic, structural, or logistical details toallow the drone 100 to operate in a semi-autonomous or fully-autonomousnature. When several drones 100 are used to perform a task, the flightmap 108 may include zones where each drone 100 is assigned coverage toincrease coverage and to reduce the possibility of collisions.

The sensor array 110 includes one or more sensors. Data captured by thesensor array 110 may be used internally by the drone 100 duringnavigation or externally by operators of the drone 100. Sensors mayinclude, but are not limited to, temperature sensors, pressure sensors,electro-optical sensors, infrared sensors, depth cameras, camera arrays,microphone arrays, gyroscopes, accelerometers, proximity sensors,microphones, and magnetometers.

In various examples, autonomous movement of the drone 100 isaccomplished using flight module 104 and one or more of the sensor array110, the flight hardware 106, and the flight map 108. In an example, theflight module 104 includes collision detection logic. To this end,readings from a proximity sensor in the sensor array 110 may be used todetermining how close the drone 100 is to an object (e.g., a wall oranother drone). In an example, data stored in the flight map 108 (incombination with sensor readings) is used to avoid objects. For example,the drone 100 may navigate around known locations of tall structures(e.g., cell towers, buildings) or fly to a sufficient height beforeproceeding to a location. For ground-based drones 100, the drone 100 mayavoid known areas with water, holes, or the like.

The flight module 104 may also utilize image data obtained using anelectro-optical or infrared sensor to avoid collisions with objects. Forexample, the flight module 104 may analyze image data using patternmatching algorithms to classify an object in the path of the drone 100while moving.

The flight module 104 may also be used to coordinate positioning, flightpatterns, camera angles, sensor configuration, and other aspects ofdrone operation among several drones that are acting together as a droneswarm. While FIG. 1 illustrates a single drone 100, it is understoodthat multiple drones 100 may act together to complete a task assigned bya user. Coordination among the drones in a swarm may be autonomouslyperformed by a combination of drones in the swarm (e.g., cooperativecomputing) or by a master drone of the swarm (e.g., central computing).

The communication subsystem 112 includes one or more receivers,transmitters, or transceivers to communicate with the remote controlsystem 102 over one or more networks. In an example, a controlinstruction is received by the communication subsystem 112 over thenetwork connection (e.g., network 114). The control instruction mayindicate what the next course of action is for the drone 100. Forexample, the control instruction may be a navigation instruction, ageneral directive, or other configuration setting. The communicationsubsystem 112 may relay the control instruction to the flight module 104at which point the flight module 104 implements the instruction.

A network 114 communicatively connects the drone 100 with the remotecontrol system 102. The network 114 may include local-area networks(LAN), wide-area networks (WAN), wireless networks (e.g., 802.11 orcellular network), the Public Switched Telephone Network (PSTN) network,ad hoc networks, cellular, personal area networks or peer-to-peer (e.g.,Bluetooth®, Wi-Fi Direct), or other combinations or permutations ofnetwork protocols and network types. The network 114 may include asingle local area network (LAN), a wide-area network (WAN), orcombinations of LAN's or WAN's, such as the Internet. The network 114may provide a secure, encrypted, or proprietary connection between theremote control system 102 and the drone 100.

The remote control system 102 may be a standalone device or part ofanother device. The remote control system 102 may be a smartphoneexecuting a remote control application, a tablet, a wearable device(e.g., smartglasses), a transmitter with a dual-stick layout and adisplay, a laptop, etc. The remote control system 102 may be acombination of devices, such as smartglasses for providing visualfeedback to the user and camera-based control to the drone 100 and ane-textile glove to provide the user tactile feedback from the drone 100.The e-textile glove may be used to point at potential objects ofinterest or as another user input device in various embodiments. Theremote control system 102 may also include at least one processing unit(e.g., a central processing unit, a graphical processor, anapplication-specific integrated circuit) that includes one or morecores. The at least one processing unit may execute software stored onthe remote control system 102 to perform the functions described herein.

The remote control system 102 includes a display device 116, a controluser interface 118, a control module 120, and a communication subsystem122. The display device 116 is a display on the remote control system102 that presents the control user interface 118. The control userinterface 118 may include various input controls to allow a user toconfigure the drone's (or drone swarm's) operation. Various parametersor settings may be configured by the user via the control user interface118 including, but not limited to, search parameters, footageproperties, and other settings.

Search parameters are used by the drone 100 to find objects, people, orpaths. Search parameters may include geographic locations (e.g., GPScoordinates), waypoints, paths, types of hazards to avoid, types ofbusiness to find, etc. Search parameters may be limited to a certaintask or type of task. For example, a task may be to run forwardreconnaissance for a user who is out for a jog on a wooded trail. Duringthe task, the drone 100 may be used in conjunction with other drones 100to scout the trail ahead of the user to alert the user of any hazards.Hazards for this type of task may be limited to fallen trees, muddy orflooded areas of the trail, and washed out areas of the trail.

Footage properties may include the pixel resolution, number ofperspectives of an object, storage preferences (e.g., file type, storagelocation, retention policy, etc.), image or video quality (e.g.,standard definition or high definition), whether to store theimage/video data locally, or unicast or broadcast it, or other aspectsof the video or image being captured.

User input may include touch input on the control user interface 118,voice input, hand gestures above the remote control system 102, orphysical movement or manipulation of the remote control system 102(e.g., using dual-stick controls). Detection of the input may beaccomplished using one or more sensors of the remote control system 102(not illustrated) such as accelerometers, microphones, gyroscopes, orcameras. When user input is detected, the control module 120 mayretrieve a control instruction for the drone 100 and then transmit thecontrol instruction using the communication subsystem 122 over networkconnection 114 to the drone 100.

During operation, the drone 100 may be configured for various tasks. Thetasks are generally designed to augment one or more of the user'ssenses. Tasks may be broadly grouped into two types: assisted navigationand inspection. Assisted navigation tasks are tasks where the user istravelling or moving by some mechanism, and the drone 100 or drone swarmis used to assist the user with navigation by scouting the user's pathfor obstacles, hazards, or other things that may delay or deny the userpassage. In the urban context, the physical environment constantlychanges due to a variety of factors, due to weather, motor accidents,construction, and other events. These changes may occur in relativelysmall areas in comparison to the global map. Drones may be used toprovide localized, live data at a fine resolution (e.g., street level,block by block, etc.).

In various examples, the drone 100 analyzes data in an image database todetermine how a geographic region has changed over time. For example,many things may cause a geographic region to change. These changes maycategorized in a number of ways, including but not limited to,environmental changes, infrastructure change, and traffic hazards. Insome instances a change may have multiple categories. Environmentalchanges may include changes due to weather such as mudslides,landslides, tornados, floods, avalanches, droughts, etc Infrastructurechanges may include changes in man-made structures such as bridgedeterioration, house fires, collapsed buildings, etc. Traffic hazardsmay be from changes on navigable roads due to traffic patterns changes(e.g., heavy traffic), road blockages (e.g., debris on a road), orwildlife being present on a road. Other categories may also be usedwithout departing from the scope of this disclosure.

As an example, the user may go on a run early in the morning before thesun has fully risen in an urban environment where there is unevenpavement. The drone 100 may be configured to detect a potential trippinghazard ahead of the user and send an audible or haptic alert to theuser's wearable devices, e.g., a Bluetooth headset, fitness watch,smartphone, etc. The drone 100 may also scout ahead when choosingbetween various routes to detect where there is the least amount ofobstruction on the user's route, which may be preconfigured ordynamically determined Obstructions may be people taking pets on walks,highly congested streets, construction, temporary closures, and othertemporal forces.

As another example, in an outdoor context with limited connectivity, theuser may download maps to the drone 100 to provide navigation prior tobeginning a trail run. In off-trail situations, the drone 100 may beconfigured to move ahead of the user to guide the user away frompotential hazards and impassable conditions, such as loose rocks orcliff edges. The drone 100 may also be used to direct the user back to amore heavily used trail. The drone 100 may help in situations byidentifying feasible routes across difficult terrain (e.g., ravines orcrevasses). This type of scouting is a real challenge as climbers andtrail hikers often cannot tell the best route to take. Also, due toweather or time, conditions may shift quickly and without warning,rendering known routes suddenly impassable. Real time scouting in thisinstance is very useful.

The other type of general task is an inspection task. Inspection tasksare designed to allow a person to remotely inspect an object, person, orthing using a single drone 100 or a drone swarm. Various types ofinspection may be provided, including visual, audio, tactile, orolfactory. The user may direct the drone 100 or drone swarm to aparticular object, person, or thing using various user controls. In anexample, the user is able to indicate to the drone 100 the object/areato cover, for visual and/or audio recording by touching on the controluser interface 118 the object/area to be inspected by the drone 100. Theuser may be presented a map on which the user may indicate a location orposition to inspect. Alternatively, the user may use a camera on theremote control system 100 to frame an object in a camera's viewer andthe drone 100 may use object recognition technologies, which may beaided by location/direction sensors or eye tracking, to determine theobject of interest. In an embodiment, The perspective of the output fromthe drone 100 may be correlated to the user's visual or auditoryperspective. This may be enabled, for example, by the user wearing oneor more of an eye tracker, head-mounted compass, or head-mountedmicrophone array to determine the direction of the user's gaze orrelative position to a sound in the environment that is being recordedby the drones 100. The drone 100 may provide a presentation via theoutput device 124 of the drone swarm to correlate with the user'sperspective relative to the visual or auditory source in theenvironment.

Once an object of interest has been identified and transmitted to thedrone 100 or the drone swarm, the drone(s) 100 may leverage the ninedegrees of freedom sensors 110 (accelerometers, gyroscopes, andmagnetometers), object recognition, and other imaging techniques tocapture image data, audio data, tactile data, or olfactory data of theobject of interest. The drone swarm may include ground-based(terrestrial) robots that may obtain camera angles that may be difficultfor aerial drones, and vice versa. Also, some drones in a drone swarmmay have distinct or particular sensors that other drones of the droneswarm do not have. This may due to payload capacities or battery limits,or other operational limits on single drones. The drone swarm, theremote control system 102, or another system may coordinate the multipleinput streams from the drone swarm.

FIGS. 2-4 are flowcharts illustrating control and data flow duringoperation, according to an embodiment. At state 202, an interactionbetween a user and a drone begins. The user selects a path of travel andactivates the drone for guidance (operation 204). The user chooses alertpreferences, e.g., what the drone should alert the user of, such asparking spots, uneven pavement, a donut shop, etc. (operation 206). Thedrone deploys (operation 208) and begins scanning an area. At decisionblock 210, it is determined whether more drones are needed. This may bedue to needing a finer level of granularity, needing to scan a largearea in near real time, or to bring other sensors, camera perspectives,or operational modes into the drone swarm's capabilities. If additionaldrones are needed, then at decision block 212 it is determined whetherthere are additional drones to be deployed. Drones may be supplied by acommunity pool or from a private pool. Communities may provide dronesfor residents, for example. Otherwise, a private citizen may own andoperate a fleet of drones to be used in different situations.Alternatively, private and public drones may be recruited and worktogether.

If there are additional drones available, then at operation 214 theadditional drones are deployed and the drone swarm coordinates flightpaths. This coordination may be performed using cooperative planning ordistributed planning Cooperative planning may be performed amongst thedrones in the drone swarm by each drone sharing its location, flightpath, remaining battery life, current task or subtask, capabilities(e.g., available sensors), direction sensors are facing, and the likewith other drones in the swarm. Based on each other's abilities,location, and other factors, each drone may intelligently decide to takean action or a series of actions. When a drone determines an action orseries of actions, it may inform the other drones in the swarm, whichmay affect the pending or available actions that other drones mayperform.

Alternatively, in a distributed drone planning scheme, a single drone(or a user) may identify a task or subtask for each drone in the swarm.Distributed drone planning and control may be easier to implement due toreduced communication and control routines used, but may be awkward tomaintain control for a user who wants to concentrate on another task(e.g., trail running) Thus, different configuration modes may beprovided for different user bases.

In either a cooperative or distributed planning scheme, when one droneis disabled (e.g., low battery, damaged due to a collision, etc.),another drone or drones may cover for the disabled drone. The disableddrone's responsibilities may be distributed among the remaining dronesin the drone swarm or a new drone may be called in to replace thedisabled drone. Sensor data, images, videos, or other information may becommunicated from the disabled drone to one or more drones in the droneswarm to preserve the information and communicate the information backto the user.

At operation 216, the path is optimized to ensure that there issufficient coverage for the user's path. With a single drone, thecoverage may be a relatively narrow cone, such as a forward cone of 15degrees coverage. With more drones, a wider forward cone may be covered.With enough drones, a full 360 degrees coverage may be provided to scoutthe user's forward and rearward direction to alert the user of hazardsthat the user is approaching (e.g., a rock fall that is blocking thetrail ahead) and that are approaching from the rear (e.g., a biker onthe trail approaching from behind).

Periodically or regularly, the drone determines whether the task iscomplete (decision block 218). When the task is complete, the dronereturns to base (operation 220) to recharge or be made available for anew deployment with the user or another user. When the task is not yetcomplete, the drone (or drones) continue on task (operation 222) and theflow returns to 206, where the parameters are rechecked and may be usedto define a new aspect of the task, which may involve the use of more orfewer drones.

FIGS. 3 and 4 illustrate the drone control and data flow, according toembodiments. In FIG. 3, when a user selects a path of travel (operation204), a maps application programming interface (API) is invoked andprovides information to the drone (operation 302). The drone uses thepath of travel provided by the user and the maps API to predict the usermovement and initiate drone movement based on predicted user movement(operation 304). The drone uses computer vision and depth analysis toanalyze a video feed obtained from one or more cameras on the drone(operation 306). The drone predicts the user's path (operation 308).This may be performed primarily based on the user's selected path oftravel. This may also be performed by tracking the user's previouslychosen path in a given area. This may also be performed by analyzing theuser's current direction, speed, and capabilities. For example, a userin a wheelchair may not be expected to travel over rocky terrain orclimb steep inclines. Thus a relatively flat portion of an off-roadtrail may be identified as the predicted user path over a steep inclinedportion.

As the user traverses the path of travel, the drone scans for potentialhazards, obstructions, or changes along the route (operation 310). Whenthe drone identifies a potential hazard, obstruction, or change, thedrone transmits an alert to the user (operation 312). The alert may beprovided using one or more mechanisms, such as via Bluetooth™headphones, a smartwatch, a smart phone, or the like, using audio,haptic, or visual alert modes (or combinations thereof). Alerts may bepresented on a user device (e.g., a smartwatch or the drone removecontrol device) or from one or more drones in the drone swarm. Thealerts may be configured by the user as notification preferences.

A new route may be chosen for the user (operation 314) and communicatedto the user. One or more routes may be presented. The user may beprovided a user interface to choose a route selection or confirm a routechange. If the user's route is completed (decision block 316), then theuser finishes the route (state 318) and the interaction ends. If theroute is not finished, then the user continues on the route (state 320)and the control and data flow continue to operation 302.

FIG. 4 illustrates a similar control and data flow as that in FIG. 3,with the difference that when hazards, obstructions, or changes alongthe route are detected (operation 310 of FIG. 3), additional drones areused to improve mapping granularity, geographic coverage, or toimplement additional sensors (operation 402). The addition of drones maybe based on the type of hazard, obstruction, or change along the route.For example, when a tree has fallen and blocks the hiker's path, then asingle drone may be able to scout an alternative path and suggest analternative route. In contrast, when the obstacle or hazard is larger,such as a forest fire, then additional drone may be used to scout theperimeter of the fire zone and suggest alternative routes. The use ofadditional drones may also be necessary to expedite the re-routing whena fast-moving fire is involved.

Hazard, obstacle, and route changes in operation 310 may be based onenvironmental factors, such as low light or shadows. For example, aslight fades, a user who is trail running may be in more danger oftripping or falling on portions of trail that are too dark. Severaldrones may be used to ensure that the user is informed of dark,dangerous areas of the trail. Terrestrial drones may be used for areaswhere trees and other objects may make it difficult for a flying droneto safely scout. As another example, as daylight fades, a drone may beconfigured to scan for a suitable camp site that is within a certainarea around the user based on the user's hiking speed, remaining light,lighting conditions, trail location, water availability (e.g., a nearbystream), etc.

FIG. 5 is a diagram illustrating operation of a drone, according to anembodiment. From a control perspective, the user is able to indicate tothe drone the object or area to cover, investigate, or monitor. The usermay view an object 500 with a camera 502, which may be integrated into auser device (e.g., a smartphone, camcorder, etc.). Based on the field ofview of the camera 502 (e.g., image, location, direction, perspective,etc.), a drone 504 is instructed to focus on the object 500 or toperform a certain action, such as a trajectory. The user of the camera502 may interact with a display (e.g., a smartphone screen) and provideinput on the image of the object 500 as captured by the camera 502,which is translated and transposed to the drone 504 to provide motioncontrol instruction or other instructions to the drone 504.

FIG. 6 is a block diagram illustrating a personal sensory drone system600, according to an embodiment. The personal sensory drone system 600includes a drone remote control system 602, which includes a task module604, a transceiver 606, and a user interface module 608. The task module604 may be configured to transmit a task to a drone swarm for the droneswarm to execute, the drone swarm including at least two drones. In anembodiment, the drone swarm includes at least one aerial drone and atleast one terrestrial drone.

The drone remote control system 602 may include multiple physicaldevices, such as a smartwatch to communicate a map to the drone swarmand a Bluetooth™ headset to receive audible alerts from the drone swarm.

The transceiver 606 may be configured to receive information from thedrone swarm related to the task. In an embodiment, the task comprises anassisted navigation task, and the transceiver is to transmit a set ofparameters to the drone swarm to configure the assisted navigation task.In a further embodiment, the set of parameters includes user preferencesdefining a set of hazards to avoid while traversing a path, and thedrone swarm scouts the path to identify a hazard from the set ofhazards. In a further embodiment, the set of hazards includes a hole, acliff, or a tripping hazard.

The user interface module 608 may be configured to present a userinterface based on the information received from the drone swarm.

In an embodiment, the drone remote control system 602 comprises anavigation module to determine an alternative route and present thealternative route to the user via the user interface. In an embodiment,the drone swarm is to determine an alternative route and communicate thealternative route to the user via the user interface.

In an embodiment, the drone remote control system 602 comprises anavigation module to transmit a geographical map to the drone swarm. Thegeographical map may be used by the drone swarm for navigation andanalyzing routes. The user may optionally upload geographical maps,which may be useful when a drone is unable to connect to a wide-areanetwork to acquire its own maps.

In an embodiment, the task comprises an inspection task, and the userinterface module 608 is to transmit an object of interest to the droneswarm, where the information received from the drone swarm includesvisual data, audio data, tactile data, or olfactory data. The object ofinterest may be any type of object including structures, buildings,people, geographic areas, vehicles, or the like. An inspection task maydirect a drone swarm to approach a stationery object of interest (e.g.,a statue on the top of a building) or follow a moving object of interest(e.g., a child).

In an embodiment, to transmit the object of interest to the drone swarm,the user interface module 608 is to transmit an image of the object ofinterest to the drone swarm, where the drone swarm uses image analysisto identify the object of interest in a field of view of a cameramounted on at least one drone of the drone swarm. For example, the usermay take a picture of an object of interest and transmit the picture tothe drone swarm. The drone swarm may use image analysis and objectrecognition to navigate to the object in the image and inspect it.

In an embodiment, to transmit the object of interest to the drone swarm,the user interface module 608 is to transmit a geolocation to the droneswarm. The geolocation may include latitude and longitude coordinates,along with an altitude to provide a three-dimensional point in space.Alternatively, the geolocation may only include the latitude andlongitude. Various geographical navigation systems may be used, such asGlobal Positioning System (GPS) or Global Navigation Satellite System(GLONASS). In an embodiment, the geolocation comprises globalpositioning system coordinates.

In an embodiment, the information received from the drone swarm includesvisual data from multiple drones in the drone swarm, and the userinterface module 608 is to composite the visual data from the multipledrones in the drone swarm to produce composited visual data and presentthe composited visual data to the user. The multiple images may be fromdifferent perspectives, using different image capture techniques, or usedifferent fields of view. Other multi-image capture techniques may beused.

In an embodiment, to present the composited visual data, the userinterface module 608 is to present multiple views of the object ofinterest from the respective multiple drones in the drone swarm. Thesemay be presented like multiple camera feeds with one main windowpresenting the active view and user interface controls to allow the userto swap between active video feeds, one from each active drone. Asdiscussed above, the relative camera perspective provided by the droneswarm may be correlated to the user's viewing perspective.

In an embodiment, to present the composited visual data, the userinterface module 608 is to present an overlay of image data from themultiple drones. For example, one drone may capture infrared images andanother drone may capture visible light images. The user may optionallyactivate and deactivate one or more layers (visible, infrared) to viewthe object of interest in several views.

In an embodiment, to present the composited visual data, the userinterface module 608 is to build a three-dimensional model of the objectof interest from the visual data provided from the multiple drones. Withenough visual information, the object of interest may be rendered as athree-dimensional object. A user with a virtual reality system may theninteract with the three-dimensional object in virtual space.

In an embodiment, to present the composited visual data, the userinterface module 608 is to present a view of the object of interest thatcorresponds to a viewing perspective of the user.

In an embodiment, the information received from the drone swarm includesaudio data from multiple drones in the drone swarm, and the userinterface module 608 is to select an audio signal from one of themultiple drones in the drone swarm, the audio signal having thestrongest signal-to-noise ratio of available audio signals from themultiple drones in the drone swarm.

In an embodiment, the user interface module 608 is to combine the audiosignal from one of the drones of the drone swarm with a video signalprovided by the drone controller, to produce an audio-visualpresentation; and present the audio-visual presentation in the userinterface. For example the drone controller may have a world-facingcamera so that the user may act as the videographer. However, in somesituations, e.g., long range surveillance of a bird, the user may not beclose enough to the object of interest to hear. The drone may act as along range microphone in such a situation.

In an embodiment, the information received from the drone swarm includestactile data from at least one drone in the drone swarm, and the userinterface module 608 is to coordinate a tactile response with a wearabledevice being worn by the user to recreate a tactile feel of the objectof interest. The tactile information may be acquired using a physicalfeeler or antenna that is used to sweep the surface of the object ofinterest. Alternatively or in combination with physical dataacquisition, a drone may use high-resolution imaging and other analysisto determine a texture, firmness, or other tactile information of theobject of interest. The user may be able to remotely feel an objectusing equipment that provides a tactile user interface. One such pieceof equipment is a glove with haptic feedback mechanism, such askinesthetic force and vibrotactile feedback. In an embodiment, thewearable device comprises a glove.

In an embodiment, the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, and topresent the user interface based on the information received from thedrone swarm, the user interface module 608 is to present arepresentation of a smell corresponding with the olfactory data. In anembodiment, the representation of the smell is a chemical name,structure, or indication. The indications may be a picture of an objectthat produces a similar or same smell (e.g., rotten eggs for sulfur).

Using a small vacuum and an airtight, sterile canister, for example, adrone in the drone swarm may capture a small amount of ambient air. Theambient air may be tested using various filters, microscopes, chemicals,or the like. Alternatively, the drone may deliver the air sample to theuser for testing, smelling, or the like. In an embodiment, theinformation received from the drone swarm includes olfactory data fromat least one drone in the drone swarm, the olfactory data being a sampleof the air around the object of interest.

In an embodiment, the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, and the droneremote control system 602 is to identify an allergy that the user has,and to present the user interface, the user interface module 608 is topresent a warning to the user that the object of interest may be anallergen. The user may indicate allergens in a user preferences orcontrol section, which may then be referenced when testing an airsample. Allergen testing may be performed by using microscopes, chemicaltesting, etc. For example, the drone may capture a small amount ofambient air around the object of interest, the air may be vacuumedthrough one or more filters, each with smaller passageways to captureparticulate matter. Then using a microscope (or other optical technique)and an image library, the drone may identify particulates that matchallergens in the image library. Particle counts for pollen or mold maybe determined using a similar mechanism or by using other mechanisms,such as an adhesive surface to capture pollen and mold, the microscopicanalysis of the adhesive surface to estimate counts.

FIG. 7 is a flowchart illustrating a method 700 of implementing apersonal sensory drone system, according to an embodiment. At block 702,a task for a drone swarm to execute is transmitted from a dronecontroller to the drone swarm, the drone swarm including at least twodrones. In an embodiment, the drone swarm includes at least one aerialdrone and at least one terrestrial drone.

In an embodiment, the task comprises an assisted navigation task, andthe method further comprises transmitting a set of parameters to thedrone swarm to configure the assisted navigation task. In a furtherembodiment, the set of parameters includes user preferences defining aset of hazards to avoid while traversing a path, and the drone swarmscouts the path to identify a hazard from the set of hazards. In afurther embodiment, the set of hazards includes a hole, a cliff, or atripping hazard.

At block 704, information is received from the drone swarm related tothe task. At block 706, a user interface is presented based on theinformation received from the drone swarm.

In an embodiment, the method 700 includes determining an alternativeroute and presenting the alternative route to the user in the userinterface. In a further embodiment, determining the alternative route isperformed by the drone swarm.

In an embodiment, the method 700 includes transmitting a geographicalmap to the drone swarm.

In an embodiment, the task comprises an inspection task, and the method700 further comprises transmitting an object of interest to the droneswarm, where the information received from the drone swarm includesvisual data, audio data, tactile data, or olfactory data. In anembodiment, transmitting the object of interest to the drone swarmcomprises transmitting an image of the object of interest to the droneswarm, the drone swarm uses image analysis to identify the object ofinterest in a field of view of a camera mounted on at least one drone ofthe drone swarm.

In an embodiment, transmitting the object of interest to the drone swarmcomprises transmitting a geolocation to the drone swarm. In anembodiment, the geolocation comprises global positioning systemcoordinates.

In an embodiment, the information received from the drone swarm includesvisual data from multiple drones in the drone swarm, and the method 700further comprises compositing the visual data from the multiple dronesin the drone swarm to produce composited visual data and presenting thecomposited visual data to the user. In a further embodiment, presentingthe composited visual data comprises presenting multiple views of theobject of interest from the respective multiple drones in the droneswarm. In another embodiment, presenting the composited visual datacomprises presenting an overlay of image data from the multiple drones.In another embodiment, presenting the composited visual data comprisesbuilding a three-dimensional model of the object of interest from thevisual data provided from the multiple drones.

In an embodiment, presenting the composited visual data comprisespresenting a view of the object of interest that corresponds to aviewing perspective of the user.

In an embodiment, the information received from the drone swarm includesaudio data from multiple drones in the drone swarm, and the method 700further comprises selecting an audio signal from the multiple drones ofthe drone swarm, the audio signal having the strongest signal-to-noiseratio of available audio signals from the multiple drones in the droneswarm. In an embodiment, the method 700 includes combining the audiosignal from one of the drones of the drone swarm with a video signalprovided by the drone controller, to produce an audio-visualpresentation; and where presenting the user interface based on theinformation received from the drone swarm comprises presenting theaudio-visual presentation in the user interface.

In an embodiment, the information received from the drone swarm includestactile data from at least one drone in the drone swarm, and the method700 further comprises coordinating a tactile response with a wearabledevice being worn by the user to recreate a tactile feel of the objectof interest. In an embodiment, the wearable device comprises a glove.

In an embodiment, the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, andpresenting the user interface based on the information received from thedrone swarm comprises presenting a representation of a smellcorresponding with the olfactory data. In an embodiment, therepresentation of the smell is a chemical name, structure, orindication.

In an embodiment, the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, the olfactorydata being a sample of the air around the object of interest.

In an embodiment, the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, and themethod further comprises identifying an allergy that the user has, andpresenting the user interface comprises presenting a warning to the userthat the object of interest may be an allergen.

Embodiments may be implemented in one or a combination of hardware,firmware, and software. Embodiments may also be implemented asinstructions stored on a machine-readable storage device, which may beread and executed by at least one processor to perform the operationsdescribed herein. A machine-readable storage device may include anynon-transitory mechanism for storing information in a form readable by amachine (e.g., a computer). For example, a machine-readable storagedevice may include read-only memory (ROM), random-access memory (RAM),magnetic disk storage media, optical storage media, flash-memorydevices, and other storage devices and media.

Examples, as described herein, may include, or may operate on, logic ora number of components, modules, or mechanisms. Modules may be hardware,software, or firmware communicatively coupled to one or more processorsin order to carry out the operations described herein. Modules may behardware modules, and as such modules may be considered tangibleentities capable of performing specified operations and may beconfigured or arranged in a certain manner. In an example, circuits maybe arranged (e.g., internally or with respect to external entities suchas other circuits) in a specified manner as a module. In an example, thewhole or part of one or more computer systems (e.g., a standalone,client or server computer system) or one or more hardware processors maybe configured by firmware or software (e.g., instructions, anapplication portion, or an application) as a module that operates toperform specified operations. In an example, the software may reside ona machine-readable medium. In an example, the software, when executed bythe underlying hardware of the module, causes the hardware to performthe specified operations. Accordingly, the term hardware module isunderstood to encompass a tangible entity, be that an entity that isphysically constructed, specifically configured (e.g., hardwired), ortemporarily (e.g., transitorily) configured (e.g., programmed) tooperate in a specified manner or to perform part or all of any operationdescribed herein. Considering examples in which modules are temporarilyconfigured, each of the modules need not be instantiated at any onemoment in time. For example, where the modules comprise ageneral-purpose hardware processor configured using software; thegeneral-purpose hardware processor may be configured as respectivedifferent modules at different times. Software may accordingly configurea hardware processor, for example, to constitute a particular module atone instance of time and to constitute a different module at a differentinstance of time. Modules may also be software or firmware modules,which operate to perform the methodologies described herein.

FIG. 8 is a block diagram illustrating a machine in the example form ofa computer system 800, within which a set or sequence of instructionsmay be executed to cause the machine to perform any one of themethodologies discussed herein, according to an example embodiment. Inalternative embodiments, the machine operates as a standalone device ormay be connected (e.g., networked) to other machines. In a networkeddeployment, the machine may operate in the capacity of either a serveror a client machine in server-client network environments, or it may actas a peer machine in peer-to-peer (or distributed) network environments.The machine may be an onboard vehicle system, set-top box, wearabledevice, personal computer (PC), a tablet PC, a hybrid tablet, a personaldigital assistant (PDA), a mobile telephone, or any machine capable ofexecuting instructions (sequential or otherwise) that specify actions tobe taken by that machine. Further, while only a single machine isillustrated, the term “machine” shall also be taken to include anycollection of machines that individually or jointly execute a set (ormultiple sets) of instructions to perform any one or more of themethodologies discussed herein. Similarly, the term “processor-basedsystem” shall be taken to include any set of one or more machines thatare controlled by or operated by a processor (e.g., a computer) toindividually or jointly execute instructions to perform any one or moreof the methodologies discussed herein.

Example computer system 800 includes at least one processor 802 (e.g., acentral processing unit (CPU), a graphics processing unit (GPU) or both,processor cores, compute nodes, etc.), a main memory 804 and a staticmemory 806, which communicate with each other via a link 808 (e.g.,bus). The computer system 800 may further include a video display unit810, an alphanumeric input device 812 (e.g., a keyboard), and a userinterface (UI) navigation device 814 (e.g., a mouse). In one embodiment,the video display unit 810, input device 812 and UI navigation device814 are incorporated into a touch screen display. The computer system800 may additionally include a storage device 816 (e.g., a drive unit),a signal generation device 818 (e.g., a speaker), a network interfacedevice 820, and one or more sensors (not shown), such as a globalpositioning system (GPS) sensor, compass, accelerometer, or othersensor.

The storage device 816 includes a machine-readable medium 822 on whichis stored one or more sets of data structures and instructions 824(e.g., software) embodying or utilized by any one or more of themethodologies or functions described herein. The instructions 824 mayalso reside, completely or at least partially, within the main memory804, static memory 806, and/or within the processor 802 during executionthereof by the computer system 800, with the main memory 804, staticmemory 806, and the processor 802 also constituting machine-readablemedia.

While the machine-readable medium 822 is illustrated in an exampleembodiment to be a single medium, the term “machine-readable medium” mayinclude a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more instructions 824. The term “machine-readable medium”shall also be taken to include any tangible medium that is capable ofstoring, encoding or carrying instructions for execution by the machineand that cause the machine to perform any one or more of themethodologies of the present disclosure or that is capable of storing,encoding or carrying data structures utilized by or associated with suchinstructions. The term “machine-readable medium” shall accordingly betaken to include, but not be limited to, solid-state memories, andoptical and magnetic media. Specific examples of machine-readable mediainclude non-volatile memory, including but not limited to, by way ofexample, semiconductor memory devices (e.g., electrically programmableread-only memory (EPROM), electrically erasable programmable read-onlymemory (EEPROM)) and flash memory devices; magnetic disks such asinternal hard disks and removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks.

The instructions 824 may further be transmitted or received over acommunications network 826 using a transmission medium via the networkinterface device 820 utilizing any one of a number of well-knowntransfer protocols (e.g., HTTP). Examples of communication networksinclude a local area network (LAN), a wide area network (WAN), theInternet, mobile telephone networks, plain old telephone (POTS)networks, and wireless data networks (e.g., Wi-Fi, 3G, and 4G LTE/LTE-Aor WiMAX networks). The term “transmission medium” shall be taken toinclude any intangible medium that is capable of storing, encoding, orcarrying instructions for execution by the machine, and includes digitalor analog communications signals or other intangible medium tofacilitate communication of such software.

ADDITIONAL NOTES & EXAMPLES

Example 1 includes subject matter for personal sensory drones (such as adevice, apparatus, or machine) comprising: a drone remote control systemcomprising: a task module to transmit a task to a drone swarm for thedrone swarm to execute, the drone swarm including at least two drones; atransceiver to receive information from the drone swarm related to thetask; and a user interface module to present a user interface based onthe information received from the drone swarm.

In Example 2, the subject matter of Example 1 may include, wherein thedrone swarm includes at least one aerial drone and at least oneterrestrial drone.

In Example 3, the subject matter of any one of Examples 1 to 2 mayinclude, wherein the task comprises an assisted navigation task, and thetransceiver is to: transmit a set of parameters to the drone swarm toconfigure the assisted navigation task.

In Example 4, the subject matter of any one of Examples 1 to 3 mayinclude, wherein the set of parameters includes user preferencesdefining a set of hazards to avoid while traversing a path, and whereinthe drone swarm scouts the path to identify a hazard from the set ofhazards.

In Example 5, the subject matter of any one of Examples 1 to 4 mayinclude, wherein the set of hazards includes a hole, a cliff, or atripping hazard.

In Example 6, the subject matter of any one of Examples 1 to 5 mayinclude, wherein the drone remote control system comprises a navigationmodule to: determine an alternative route; and present the alternativeroute to the user via the user interface.

In Example 7, the subject matter of any one of Examples 1 to 6 mayinclude, wherein the drone swarm is to: determine an alternative route;and communicate the alternative route to the user via the userinterface.

In Example 8, the subject matter of any one of Examples 1 to 7 mayinclude, wherein the drone remote control system comprises a navigationmodule to transmit a geographical map to the drone swarm.

In Example 9, the subject matter of any one of Examples 1 to 8 mayinclude, wherein the task comprises an inspection task, and wherein theuser interface module is to: transmit an object of interest to the droneswarm, wherein the information received from the drone swarm includesvisual data, audio data, tactile data, or olfactory data.

In Example 10, the subject matter of any one of Examples 1 to 9 mayinclude, wherein to transmit the object of interest to the drone swarm,the user interface module is to: transmit an image of the object ofinterest to the drone swarm, wherein the drone swarm uses image analysisto identify the object of interest in a field of view of a cameramounted on at least one drone of the drone swarm.

In Example 11, the subject matter of any one of Examples 1 to 10 mayinclude, wherein to transmit the object of interest to the drone swarm,the user interface module is to: transmit a geolocation to the droneswarm.

In Example 12, the subject matter of any one of Examples 1 to 11 mayinclude, wherein the geolocation comprises global positioning systemcoordinates.

In Example 13, the subject matter of any one of Examples 1 to 12 mayinclude, wherein the information received from the drone swarm includesvisual data from multiple drones in the drone swarm, and wherein theuser interface module is to: composite the visual data from the multipledrones in the drone swarm to produce composited visual data; and presentthe composited visual data to the user.

In Example 14, the subject matter of any one of Examples 1 to 13 mayinclude, wherein to present the composited visual data, the userinterface module is to present multiple views of the object of interestfrom the respective multiple drones in the drone swarm.

In Example 15, the subject matter of any one of Examples 1 to 14 mayinclude, wherein to present the composited visual data, the userinterface module is to present an overlay of image data from themultiple drones.

In Example 16, the subject matter of any one of Examples 1 to 15 mayinclude, wherein to present the composited visual data, the userinterface module is to build a three-dimensional model of the object ofinterest from the visual data provided from the multiple drones.

In Example 17, the subject matter of any one of Examples 1 to 16 mayinclude, wherein to present the composited visual data, the userinterface module is to present a view of the object of interest thatcorresponds to a viewing perspective of the user.

In Example 18, the subject matter of any one of Examples 1 to 17 mayinclude, wherein the information received from the drone swarm includesaudio data from multiple drones in the drone swarm, and wherein the userinterface module is to: select an audio signal from one of the multipledrones in the drone swarm, the audio signal having the strongestsignal-to-noise ratio of available audio signals from the multipledrones in the drone swarm.

In Example 19, the subject matter of any one of Examples 1 to 18 mayinclude, wherein the user interface module is to: combine the audiosignal from one of the drones of the drone swarm with a video signalprovided by the drone controller, to produce an audio-visualpresentation; and present the audio-visual presentation in the userinterface.

In Example 20, the subject matter of any one of Examples 1 to 19 mayinclude, wherein the information received from the drone swarm includestactile data from at least one drone in the drone swarm, and wherein theuser interface module is to: coordinate a tactile response with awearable device being worn by the user to recreate a tactile feel of theobject of interest.

In Example 21, the subject matter of any one of Examples 1 to 20 mayinclude, wherein the wearable device comprises a glove.

In Example 22, the subject matter of any one of Examples 1 to 21 mayinclude, wherein the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, and whereinto present the user interface based on the information received from thedrone swarm, the user interface module is to present a representation ofa smell corresponding with the olfactory data.

In Example 23, the subject matter of any one of Examples 1 to 22 mayinclude, wherein the representation of the smell is a chemical name,structure, or indication.

In Example 24, the subject matter of any one of Examples 1 to 23 mayinclude, wherein the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, the olfactorydata being a sample of the air around the object of interest.

In Example 25, the subject matter of any one of Examples 1 to 24 mayinclude, wherein the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, and whereinthe drone remote control system is to identify an allergy that the userhas, and wherein to present the user interface, the user interfacemodule is to present a warning to the user that the object of interestmay be an allergen.

Example 26 includes subject matter for personal sensory drones (such asa method, means for performing acts, machine readable medium includinginstructions that when performed by a machine cause the machine toperforms acts, or an apparatus to perform) comprising: transmitting froma drone controller to a drone swarm, a task for the drone swarm toexecute, the drone swarm including at least two drones; receivinginformation from the drone swarm related to the task; and presenting auser interface based on the information received from the drone swarm.

In Example 27, the subject matter of Example 26 may include, wherein thedrone swarm includes at least one aerial drone and at least oneterrestrial drone.

In Example 28, the subject matter of any one of Examples 26 to 27 mayinclude, wherein the task comprises an assisted navigation task, and themethod further comprises: transmitting a set of parameters to the droneswarm to configure the assisted navigation task.

In Example 29, the subject matter of any one of Examples 26 to 28 mayinclude, wherein the set of parameters includes user preferencesdefining a set of hazards to avoid while traversing a path, and whereinthe drone swarm scouts the path to identify a hazard from the set ofhazards.

In Example 30, the subject matter of any one of Examples 26 to 29 mayinclude, wherein the set of hazards includes a hole, a cliff, or atripping hazard.

In Example 31, the subject matter of any one of Examples 26 to 30 mayinclude, determining an alternative route; and presenting thealternative route to the user in the user interface.

In Example 32, the subject matter of any one of Examples 26 to 31 mayinclude, wherein determining the alternative route is performed by thedrone swarm.

In Example 33, the subject matter of any one of Examples 26 to 32 mayinclude, transmitting a geographical map to the drone swarm.

In Example 34, the subject matter of any one of Examples 26 to 33 mayinclude, wherein the task comprises an inspection task, and wherein themethod further comprises: transmitting an object of interest to thedrone swarm, wherein the information received from the drone swarmincludes visual data, audio data, tactile data, or olfactory data.

In Example 35, the subject matter of any one of Examples 26 to 34 mayinclude, wherein transmitting the object of interest to the drone swarmcomprises: transmitting an image of the object of interest to the droneswarm, wherein the drone swarm uses image analysis to identify theobject of interest in a field of view of a camera mounted on at leastone drone of the drone swarm.

In Example 36, the subject matter of any one of Examples 26 to 35 mayinclude, wherein transmitting the object of interest to the drone swarmcomprises: transmitting a geolocation to the drone swarm.

In Example 37, the subject matter of any one of Examples 26 to 36 mayinclude, wherein the geolocation comprises global positioning systemcoordinates.

In Example 38, the subject matter of any one of Examples 26 to 37 mayinclude, wherein the information received from the drone swarm includesvisual data from multiple drones in the drone swarm, and wherein themethod further comprises: compositing the visual data from the multipledrones in the drone swarm to produce composited visual data; andpresenting the composited visual data to the user.

In Example 39, the subject matter of any one of Examples 26 to 38 mayinclude, wherein presenting the composited visual data comprisespresenting multiple views of the object of interest from the respectivemultiple drones in the drone swarm.

In Example 40, the subject matter of any one of Examples 26 to 39 mayinclude, wherein presenting the composited visual data comprisespresenting an overlay of image data from the multiple drones.

In Example 41, the subject matter of any one of Examples 26 to 40 mayinclude, wherein presenting the composited visual data comprisesbuilding a three-dimensional model of the object of interest from thevisual data provided from the multiple drones.

In Example 42, the subject matter of any one of Examples 26 to 41 mayinclude, wherein presenting the composited visual data comprisespresenting a view of the object of interest that corresponds to aviewing perspective of the user.

In Example 43, the subject matter of any one of Examples 26 to 42 mayinclude, wherein the information received from the drone swarm includesaudio data from multiple drones in the drone swarm, and wherein themethod further comprises: selecting an audio signal from the multipledrones of the drone swarm, the audio signal having the strongestsignal-to-noise ratio of available audio signals from the multipledrones in the drone swarm.

In Example 44, the subject matter of any one of Examples 26 to 43 mayinclude, combining the audio signal from one of the drones of the droneswarm with a video signal provided by the drone controller, to producean audio-visual presentation; and wherein presenting the user interfacebased on the information received from the drone swarm comprisespresenting the audio-visual presentation in the user interface.

In Example 45, the subject matter of any one of Examples 26 to 44 mayinclude, wherein the information received from the drone swarm includestactile data from at least one drone in the drone swarm, and wherein themethod further comprises: coordinating a tactile response with awearable device being worn by the user to recreate a tactile feel of theobject of interest.

In Example 46, the subject matter of any one of Examples 26 to 45 mayinclude, wherein the wearable device comprises a glove.

In Example 47, the subject matter of any one of Examples 26 to 46 mayinclude, wherein the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, and whereinpresenting the user interface based on the information received from thedrone swarm comprises presenting a representation of a smellcorresponding with the olfactory data.

In Example 48, the subject matter of any one of Examples 26 to 47 mayinclude, wherein the representation of the smell is a chemical name,structure, or indication.

In Example 49, the subject matter of any one of Examples 26 to 48 mayinclude, wherein the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, the olfactorydata being a sample of the air around the object of interest.

In Example 50, the subject matter of any one of Examples 26 to 49 mayinclude, wherein the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, and whereinthe method further comprises identifying an allergy that the user has,and wherein presenting the user interface comprises presenting a warningto the user that the object of interest may be an allergen.

Example 51 includes at least one machine-readable medium includinginstructions, which when executed by a machine, cause the machine toperform operations of any of the Examples 26-50.

Example 52 includes an apparatus comprising means for performing any ofthe Examples 26-50.

Example 53 includes subject matter for personal sensory drones (such asa device, apparatus, or machine) comprising: means for transmitting froma drone controller to a drone swarm, a task for the drone swarm toexecute, the drone swarm including at least two drones; means forreceiving information from the drone swarm related to the task; andmeans for presenting a user interface based on the information receivedfrom the drone swarm.

In Example 54, the subject matter of Example 53 may include, wherein thedrone swarm includes at least one aerial drone and at least oneterrestrial drone.

In Example 55, the subject matter of any one of Examples 53 to 54 mayinclude, wherein the task comprises an assisted navigation task, and theapparatus further comprises: means for transmitting a set of parametersto the drone swarm to configure the assisted navigation task.

In Example 56, the subject matter of any one of Examples 53 to 55 mayinclude, wherein the set of parameters includes user preferencesdefining a set of hazards to avoid while traversing a path, and whereinthe drone swarm scouts the path to identify a hazard from the set ofhazards.

In Example 57, the subject matter of any one of Examples 53 to 56 mayinclude, wherein the set of hazards includes a hole, a cliff, or atripping hazard.

In Example 58, the subject matter of any one of Examples 53 to 57 mayinclude, means for determining an alternative route; and means forpresenting the alternative route to the user in the user interface.

In Example 59, the subject matter of any one of Examples 53 to 58 mayinclude, wherein the means for determining the alternative route isperformed by the drone swarm.

In Example 60, the subject matter of any one of Examples 53 to 59 mayinclude, means for transmitting a geographical map to the drone swarm.

In Example 61, the subject matter of any one of Examples 53 to 60 mayinclude, wherein the task comprises an inspection task, and wherein theapparatus further comprises: means for transmitting an object ofinterest to the drone swarm, wherein the information received from thedrone swarm includes visual data, audio data, tactile data, or olfactorydata.

In Example 62, the subject matter of any one of Examples 53 to 61 mayinclude, wherein the means for transmitting the object of interest tothe drone swarm comprise: means for transmitting an image of the objectof interest to the drone swarm, wherein the drone swarm uses imageanalysis to identify the object of interest in a field of view of acamera mounted on at least one drone of the drone swarm.

In Example 63, the subject matter of any one of Examples 53 to 62 mayinclude, wherein the means for transmitting the object of interest tothe drone swarm comprise: means for transmitting a geolocation to thedrone swarm.

In Example 64, the subject matter of any one of Examples 53 to 63 mayinclude, wherein the geolocation comprises global positioning systemcoordinates.

In Example 65, the subject matter of any one of Examples 53 to 64 mayinclude, wherein the information received from the drone swarm includesvisual data from multiple drones in the drone swarm, and wherein theapparatus further comprises: means for compositing the visual data fromthe multiple drones in the drone swarm to produce composited visualdata; and means for presenting the composited visual data to the user.

In Example 66, the subject matter of any one of Examples 53 to 65 mayinclude, wherein the means for presenting the composited visual datacomprise means for presenting multiple views of the object of interestfrom the respective multiple drones in the drone swarm.

In Example 67, the subject matter of any one of Examples 53 to 66 mayinclude, wherein the means for presenting the composited visual datacomprise means for presenting an overlay of image data from the multipledrones.

In Example 68, the subject matter of any one of Examples 53 to 67 mayinclude, wherein the means for presenting the composited visual datacomprise means for building a three-dimensional model of the object ofinterest from the visual data provided from the multiple drones.

In Example 69, the subject matter of any one of Examples 53 to 68 mayinclude, wherein the means for presenting the composited visual datacomprise means for presenting a view of the object of interest thatcorresponds to a viewing perspective of the user.

In Example 70, the subject matter of any one of Examples 53 to 69 mayinclude, wherein the information received from the drone swarm includesaudio data from multiple drones in the drone swarm, and wherein theapparatus further comprises: means for selecting an audio signal fromthe multiple drones of the drone swarm, the audio signal having thestrongest signal-to-noise ratio of available audio signals from themultiple drones in the drone swarm.

In Example 71, the subject matter of any one of Examples 53 to 70 mayinclude, means for combining the audio signal from one of the drones ofthe drone swarm with a video signal provided by the drone controller, toproduce an audio-visual presentation; and wherein the means forpresenting the user interface based on the information received from thedrone swarm comprise means for presenting the audio-visual presentationin the user interface.

In Example 72, the subject matter of any one of Examples 53 to 71 mayinclude, wherein the information received from the drone swarm includestactile data from at least one drone in the drone swarm, and wherein theapparatus further comprises: means for coordinating a tactile responsewith a wearable device being worn by the user to recreate a tactile feelof the object of interest.

In Example 73, the subject matter of any one of Examples 53 to 72 mayinclude, wherein the wearable device comprises a glove.

In Example 74, the subject matter of any one of Examples 53 to 73 mayinclude, wherein the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, and whereinthe means for presenting the user interface based on the informationreceived from the drone swarm comprise means for presenting arepresentation of a smell corresponding with the olfactory data.

In Example 75, the subject matter of any one of Examples 53 to 74 mayinclude, wherein the representation of the smell is a chemical name,structure, or indication.

In Example 76, the subject matter of any one of Examples 53 to 75 mayinclude, wherein the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, the olfactorydata being a sample of the air around the object of interest.

In Example 77, the subject matter of any one of Examples 53 to 76 mayinclude, wherein the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, and whereinthe apparatus further comprises means for identifying an allergy thatthe user has, and wherein the means for presenting the user interfacecomprise means for presenting a warning to the user that the object ofinterest may be an allergen.

The above detailed description includes references to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments that may bepracticed. These embodiments are also referred to herein as “examples.”Such examples may include elements in addition to those shown ordescribed. However, also contemplated are examples that include theelements shown or described. Moreover, also contemplated are examplesusing any combination or permutation of those elements shown ordescribed (or one or more aspects thereof), either with respect to aparticular example (or one or more aspects thereof), or with respect toother examples (or one or more aspects thereof) shown or describedherein.

Publications, patents, and patent documents referred to in this documentare incorporated by reference herein in their entirety, as thoughindividually incorporated by reference. In the event of inconsistentusages between this document and those documents so incorporated byreference, the usage in the incorporated reference(s) are supplementaryto that of this document; for irreconcilable inconsistencies, the usagein this document controls.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one, independent of any otherinstances or usages of “at least one” or “one or more.” In thisdocument, the term “or” is used to refer to a nonexclusive or, such that“A or B” includes “A but not B,” “B but not A,” and “A and B,” unlessotherwise indicated. In the appended claims, the terms “including” and“in which” are used as the plain-English equivalents of the respectiveterms “comprising” and “wherein.” Also, in the following claims, theterms “including” and “comprising” are open-ended, that is, a system,device, article, or process that includes elements in addition to thoselisted after such a term in a claim are still deemed to fall within thescope of that claim. Moreover, in the following claims, the terms“first,” “second,” and “third,” etc. are used merely as labels, and arenot intended to suggest a numerical order for their objects.

The above description is intended to be illustrative, and notrestrictive. For example, the above-described examples (or one or moreaspects thereof) may be used in combination with others. Otherembodiments may be used, such as by one of ordinary skill in the artupon reviewing the above description. The Abstract is to allow thereader to quickly ascertain the nature of the technical disclosure. Itis submitted with the understanding that it will not be used tointerpret or limit the scope or meaning of the claims. Also, in theabove Detailed Description, various features may be grouped together tostreamline the disclosure. However, the claims may not set forth everyfeature disclosed herein as embodiments may feature a subset of saidfeatures. Further, embodiments may include fewer features than thosedisclosed in a particular example. Thus, the following claims are herebyincorporated into the Detailed Description, with a claim standing on itsown as a separate embodiment. The scope of the embodiments disclosedherein is to be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled.

What is claimed is:
 1. A personal sensory drone system, the systemcomprising: a drone remote control system comprising: a task module totransmit a task to a drone swarm for the drone swarm to execute, thedrone swarm including at least two drones; a transceiver to receiveinformation from the drone swarm related to the task; and a userinterface module to present a user interface based on the informationreceived from the drone swarm.
 2. The system of claim 1, wherein thedrone swarm includes at least one aerial drone and at least oneterrestrial drone.
 3. The system of claim 1, wherein the task comprisesan assisted navigation task, and the transceiver is to: transmit a setof parameters to the drone swarm to configure the assisted navigationtask.
 4. The system of claim 3, wherein the set of parameters includesuser preferences defining a set of hazards to avoid while traversing apath, and wherein the drone swarm scouts the path to identify a hazardfrom the set of hazards.
 5. The system of claim 1, wherein the taskcomprises an inspection task, and wherein the user interface module isto: transmit an object of interest to the drone swarm, wherein theinformation received from the drone swarm includes visual data, audiodata, tactile data, or olfactory data.
 6. The system of claim 5, whereinto transmit the object of interest to the drone swarm, the userinterface module is to: transmit an image of the object of interest tothe drone swarm, wherein the drone swarm uses image analysis to identifythe object of interest in a field of view of a camera mounted on atleast one drone of the drone swarm.
 7. The system of claim 5, whereinthe information received from the drone swarm includes visual data frommultiple drones in the drone swarm, and wherein the user interfacemodule is to: composite the visual data from the multiple drones in thedrone swarm to produce composited visual data; and present thecomposited visual data to the user.
 8. The system of claim 7, wherein topresent the composited visual data, the user interface module is topresent multiple views of the object of interest from the respectivemultiple drones in the drone swarm.
 9. The system of claim 7, wherein topresent the composited visual data, the user interface module is topresent an overlay of image data from the multiple drones.
 10. Thesystem of claim 7, wherein to present the composited visual data, theuser interface module is to build a three-dimensional model of theobject of interest from the visual data provided from the multipledrones.
 11. The system of claim 7, wherein to present the compositedvisual data, the user interface module is to present a view of theobject of interest that corresponds to a viewing perspective of theuser.
 12. A method of implementing a personal sensory drone system, themethod comprising: transmitting from a drone controller to a droneswarm, a task for the drone swarm to execute, the drone swarm includingat least two drones; receiving information from the drone swarm relatedto the task; and presenting a user interface based on the informationreceived from the drone swarm.
 13. The method of claim 12, wherein thetask comprises an assisted navigation task, and the method furthercomprises: transmitting a set of parameters to the drone swarm toconfigure the assisted navigation task.
 14. The method of claim 13,wherein the set of parameters includes user preferences defining a setof hazards to avoid while traversing a path, and wherein the drone swarmscouts the path to identify a hazard from the set of hazards.
 15. Themethod of claim 12, wherein the task comprises an inspection task, andwherein the method further comprises: transmitting an object of interestto the drone swarm, wherein the information received from the droneswarm includes visual data, audio data, tactile data, or olfactory data.16. At least one machine-readable medium including instructions, whichwhen executed by a machine, cause the machine to: transmit from a dronecontroller to a drone swarm, a task for the drone swarm to execute, thedrone swarm including at least two drones; receive information from thedrone swarm related to the task; and present a user interface based onthe information received from the drone swarm.
 17. The at least onemachine-readable medium of claim 16, wherein the task comprises anassisted navigation task, and the instructions further compriseinstructions to: transmit a set of parameters to the drone swarm toconfigure the assisted navigation task.
 18. The at least onemachine-readable medium of claim 17, wherein the set of parametersincludes user preferences defining a set of hazards to avoid whiletraversing a path, and wherein the drone swarm scouts the path toidentify a hazard from the set of hazards.
 19. The at least onemachine-readable medium of claim 16, wherein the task comprises aninspection task, and wherein the instructions further compriseinstructions to: transmit an object of interest to the drone swarm,wherein the information received from the drone swarm includes visualdata, audio data, tactile data, or olfactory data.
 20. The at least onemachine-readable medium of claim 19, wherein the information receivedfrom the drone swarm includes visual data from multiple drones in thedrone swarm, and wherein the instructions further comprise instructionsto: composite the visual data from the multiple drones in the droneswarm to produce composited visual data; and present the compositedvisual data to the user.
 21. The at least one machine-readable medium ofclaim 20, wherein the instructions to present the composited visual datacomprise instructions to present multiple views of the object ofinterest from the respective multiple drones in the drone swarm.
 22. Theat least one machine-readable medium of claim 20, wherein theinstructions to present the composited visual data comprise instructionsto build a three-dimensional model of the object of interest from thevisual data provided from the multiple drones.
 23. The at least onemachine-readable medium of claim 20, wherein the instructions to presentthe composited visual data comprise instructions to present a view ofthe object of interest that corresponds to a viewing perspective of theuser.
 24. The at least one machine-readable medium of claim 19, whereinthe information received from the drone swarm includes tactile data fromat least one drone in the drone swarm, and wherein the instructionsfurther comprise instructions to: coordinate a tactile response with awearable device being worn by the user to recreate a tactile feel of theobject of interest.
 25. The at least one machine-readable medium ofclaim 19, wherein the information received from the drone swarm includesolfactory data from at least one drone in the drone swarm, and whereinthe instructions to present the user interface based on the informationreceived from the drone swarm comprise instructions to present arepresentation of a smell corresponding with the olfactory data.